System and method for the interactive display of data in a motion capture environment

ABSTRACT

A system includes an analysis system for performing an analysis and a motion capture environment interfaced with the analysis system. The motion capture system includes at least one sensor-tracker for tracking a location of a tracked object within the motion capture environment and one or more computers collectively operable to generate a virtual reality environment corresponding to the analysis.

TECHNICAL FIELD

The present invention relates to virtual reality environments.

DESCRIPTION OF THE PRIOR ART

Many systems and methodologies exist that analyze how matter reacts whenthe matter is subjected to certain conditions. For example,computational fluid dynamics is one of the branches of fluid mechanicsthat uses numerical methods and algorithms to solve and analyze problemsthat involve fluid flows. Computers are used to perform the millions ofcalculations required to simulate the interaction of fluids and gaseswith complex surfaces used in engineering. Other such systems andmethodologies include computational stress analysis, finite elementanalysis, and the like.

One particular shortcoming of such computational methodologies andsystems is in the visualization of the output data provided from thesesystems. Often, the output data exists in three dimensions. For example,output data from a computational fluid dynamics system may includethree-dimensional location data, pressure data, temperature data, andthe like. Conventional analysis systems, however, provide visualizationof the data in fewer dimensions than the data represents. For example,conventional visualization techniques provide a “picture” of the data intwo physical dimensions on a monitor, along with color codingcorresponding to levels of other conditions, such as temperature andpressure.

There are ways of controlling virtual reality environments well known inthe art; however, considerable shortcomings remain.

BRIEF DESCRIPTION OF THE DRAWINGS

The novel features believed characteristic of the invention are setforth in the appended claims. However, the invention itself, as well asa preferred mode of use, and further objectives and advantages thereof,will best be understood by reference to the following detaileddescription when read in conjunction with the accompanying drawings, inwhich the leftmost significant digit(s) in the reference numeralsdenote(s) the first figure in which the respective reference numeralsappear, wherein:

FIG. 1 is a stylized, exemplary, perspective view of an actor within astudio of a motion capture environment;

FIG. 2 is an enlarged view of the actor of FIG. 1;

FIG. 3 is a stylized, block diagram of the motion capture system of FIG.1 interfaced with an exemplary analysis system;

FIG. 4 is stylized view of an actor viewing a representation of data ina virtual reality environment;

FIG. 5 is a stylized view of the actor of FIG. 3 viewing arepresentation of data being modified by the actor in the virtualreality environment;

FIG. 6 is a stylized view of the actor of FIG. 3 viewing arepresentation of data after the data has been modified by the actor;and

FIG. 7 is a stylized, exemplary view of representation of a virtualcontrol panel within the virtual reality environment for use by theactor.

While the invention is susceptible to various modifications andalternative forms, specific embodiments thereof have been shown by wayof example in the drawings and are herein described in detail. It shouldbe understood, however, that the description herein of specificembodiments is not intended to limit the invention to the particularforms disclosed, but on the contrary, the intention is to cover allmodifications, equivalents, and alternatives falling within the spiritand scope of the invention as defined by the appended claims.

DESCRIPTION OF THE PREFERRED EMBODIMENT

Illustrative embodiments of the invention are described below. In theinterest of clarity, not all features of an actual implementation aredescribed in this specification. It will of course be appreciated thatin the development of any such actual embodiment, numerousimplementation-specific decisions must be made to achieve thedeveloper's specific goals, such as compliance with system-related andbusiness-related constraints, which will vary from one implementation toanother. Moreover, it will be appreciated that such a development effortmight be complex and time-consuming but would nevertheless be a routineundertaking for those of ordinary skill in the art having the benefit ofthis disclosure.

In the specification, reference may be made to the spatial relationshipsbetween various components and to the spatial orientation of variousaspects of components as the devices are depicted in the attacheddrawings. However, as will be recognized by those skilled in the artafter a complete reading of the present application, the devices,members, apparatuses, etc. described herein may be positioned in anydesired orientation. Thus, the use of terms such as “above,” “below,”“upper,” “lower,” or other like terms to describe a spatial relationshipbetween various components or to describe the spatial orientation ofaspects of such components should be understood to describe a relativerelationship between the components or a spatial orientation of aspectsof such components, respectively, as the device described herein may beoriented in any desired direction.

Referring to FIG. 1, in a virtual reality environment or virtual realityscene, one or more users or actors 101 interact with one or morephysical objects 103 and/or 105 in a physical or real environment and/orone or more virtual artifacts 107 and/or 109 in the virtual realityenvironment. The one or more actors 101 are physically present in athree-dimensional space, known as a studio 111 in which the one or moreactors 101 may move the one or more physical objects 103 and/or 105. Amotion capture environment 113 is contained by studio 111. Motioncapture environment 113 includes one or more computers 115 and softwareresident on the one or more computers 115 that are operable to generatevirtual reality scenes. Motion capture environment 113 further includesa framework 117, upon which to mount tracker-sensors 119 and/ortracker-sensor combinations, which are described in greater detailherein. The software includes one or more computer programs thatinterpret information from the tracker-sensors and one or more computerprograms that create the virtual reality scenes or environment.

A virtual representation of studio 111 exists in motion captureenvironment 113, which hosts the virtual reality environment. The one ormore actors 101 use display devices, for example, headset viewers, suchas a headset viewer 201 of FIG. 2; monitors, such as a monitor 121; orthe like, to view the virtual reality environment. The virtual realityenvironment is the scene that the one or more actors 101, or other suchobservers, see via the display devices. The virtual reality environmentmay be a virtual representation of the studio or the virtual realityenvironment may be a virtual representation of any other real orimagined three-dimensional space. Moreover, the virtual realityenvironment may be a combination of a virtual representation of thestudio and a virtual representation of another real or imaginedthree-dimensional space.

Physical objects, such as physical objects 103 and 105, that aredisposed within studio 111 and that are moved by the one or more actors101, are tracked using motion capture environment 113. These “trackedobjects” may be tracked by a variety of sensor methodologies, including,but not limited to, reflectors, such as reflectors 123 and 125 andreflector 203 of FIG. 2; inertial measurement units; and the like.Examples of such inertial measurement units include, but are not limitedto, ring laser gyroscopes, accelerometers, ultrasonic emitter-receptors,and the like. Referring to FIG. 2, examples of tracked objects include,but are not limited to, wands, such as a wand 205; gloves, such as aglove 207; hats, such as a hat 209; head mounted displays, such asheadset viewer 201; boots, such as boot 211; and the like.

Tracker-sensors, such as tracker sensors 119, interface with motioncapture environment 113 and determine where a tracked object, such asphysical objects 103 and 105, is located within the physical space ofthe studio. Such tracker-sensors may comprise a single unit or aplurality of units. The tracker-sensors may be attached to a framework,such as framework 117, which defines the physical limits of the studioor may be attached to the tracked objects, or both. Whiletracker-sensors may utilize various methodologies for tracking trackedobjects, certain tracker-sensors use inertial acceleration withsubsequent integration to provide rate and displacement information,ultrasonic measurement, optical measurement, near infrared measurement,as well as methods that use other bands of radiation within theelectromagnetic spectrum.

As shown in FIG. 3, motion capture environment 113 has an interface 301to an analysis system 303. Examples of such analysis systems include,but are not limited to, a computational fluid dynamics system, acomputational stress analysis system, a finite element analysis system,and the like. There are innumerable types of analysis systems 303 thatmay be interfaced with motion capture environment 113. Motion captureenvironment 113 generates a virtual reality environment or scene thatincludes data from analysis system 303, so that the actor, such as actor101, may interact with the virtual representation of the analysis data.Note that the analysis data may be represented in many dimensions, suchas three physical dimensions, e.g., height, length, depth; color; sound;and the like.

While interacting with the virtual representation of the analysis data,actor 101 wears a tracking costume, comprising, for example, headsetviewer 201, one or more wands 205, one or more gloves 207, hat 209, oneor more boots 211, each shown in FIG. 2, or the like. Wands 205, gloves207, hat 209, boots 211, and the like are tracked by thetracker-sensors. The process or device to be observed is generated byanalysis system 303 and the resulting geometry is sent from the analysisprogram to the virtual reality scene, created by motion captureenvironment 113, via interface 301. Actor 101 can observe the object andcan touch the object in a virtual sense. Actor 101 can reorient himselfor herself relative to the object, the object can be resized fordetailed inspection of a portion of the object or for an overallimpression. In one embodiment, these actions are accomplished via avirtual control panel 127, shown in FIGS. 1 and 7, which is discussed ingreater detail herein. Alternatively, or in conjunction with virtualcontrol panel 127, the actions may be accomplished by actual virtualmanipulation of the object by actor 101. In some cases, a virtual objectmay be modified in real time by actor 101 and the results shownimmediately.

There are innumerable implementations of the interactive display of datain motion capture environment 113. One exemplary implementation is thevisualization of a flexible beam that has unacceptably high vibrationdisplacement when exposed to a particular vibratory force. Referring nowto FIG. 4, an actor, such as actor 101, within a virtual realityenvironment observes a virtual beam 401 being subjected to certainconditions within a modal analysis system, which is one type of analysissystem 303 (shown in FIG. 3). The modal analysis system determines theresponse of virtual beam 401 to the applied conditions and sends theresulting geometrical output to the virtual reality scene. Motion ofvirtual beam 401 is depicted in the virtual reality environment in timeand space.

As depicted in FIG. 5, actor 101, if he or she so desires, may attemptto modify virtual beam 401 by calling up a virtual toolbox, such as avirtual toolbox 501, that contains, for example, virtual tuning weights.Actor 101 may select one of the virtual tuning weights and place it onvirtual beam 401. Actor 101 may request a new modal analysis.Preferably, the results of the new analysis display immediately, as ifactor 101 had actually, physically placed a real, physical weight 601 ona real vibrating beam 603, as shown in FIG. 6. If the weight isinsufficient or placed incorrectly, actor 101 can continue to iteratethe mass and placement of the weight until the vibration levels areacceptable.

Referring now to FIGS. 1 and 7, a virtual control panel, such as thedisplayed representation of virtual control panel 127, also known as asynthetic remote control, exists as a virtual artifact only in thevirtual reality environment and is produced by motion captureenvironment 113. Virtual control panel 127 is a virtual object displayedby the display device, such as headset viewer 201 of FIG. 2, used byactor 101 to see the virtual reality environment. Virtual control panel127 may also be displayed on other display devices, such as monitor 121of FIG. 1, that can be viewed by those that are not actors. In oneembodiment, virtual control panel 127 is a virtual means for inputtinginformation to motion capture environment 113 by actor 101. For example,as shown in FIG. 7, virtual control panel 127 comprises a plurality ofcontrols that may be manipulated by actor 101. In the embodimentillustrated in FIG. 7, the controls include, but are not limited to, forexample, buttons 701, 703, and 705; switches 707 and 709; and knobs 711and 713, which may be manipulated by actor 101. It should be noted thatvirtual control panel 127 may include additional or alternative controlsthat may be manipulated by actor 101.

Moreover, virtual control panel 127 may include one or more means forproviding information from motion capture environment 113 to actor 101.For example, virtual control panel 127 may provide information relatingto a simulation being performed to actor 101, such as a color scale orgraph 715 representing certain parameter levels or a textual display 716providing other such information. Moreover, virtual control panel 127may comprise other tools which can be utilized by actor 101 in thevirtual reality environment. For example, virtual control panel 127 mayprovide a virtual ruler 717, which can be used by actor 101 to measurevirtual artifacts, distances between virtual artifacts, or the like.

It should be noted that the virtual control panel is able to “float” invirtual space at a location specified by actor 101 and may be moved fromone place in the virtual environment to another place in the virtualenvironment by actor 101. The controls may be manipulated by actor 101'svirtual hand, defined by a glove, such as glove 207, best shown in FIG.2. Representations or “markers” 719, 721, 723, and 725, corresponding toa reflector from a glove worn by actor 101, are also illustrated in FIG.7. The manipulation of the control is detected by interpreting themotion of the actor's virtual hand when the actor's virtual hand is in“touching” proximity to the control, as determined by motion captureenvironment 113. Motion capture environment 113 determines how thecontrol has been manipulated and reacts to the manipulationappropriately.

In one embodiment, actor 101 in studio 111 manipulates a virtual hand inthe virtual reality environment by wearing and physically moving glove207, best shown in FIG. 2, which is a tracked object. Motion captureenvironment 113 interprets the motion of the glove and determines whereactor 101's virtual hand is located in the virtual reality environmentand how the virtual hand is oriented. In this embodiment, actor 101wears headset viewer 201, best shown in FIG. 2, that is equipped with asynthetic vision viewer. The synthetic vision viewer displays to actor101 the virtual reality environment and the location of the virtual handwithin the virtual reality environment. Thus, actor 101 can see thevirtual hand in the context of the scene of the virtual realityenvironment.

In FIGS. 1 and 2, actor 101 is wearing headset viewer 201 and glove 207.Actor 101 is reaching into empty physical space to press a button, suchas one of buttons 701, 703, or 705, of virtual control panel 127.

Virtual control panel is preferably positioned at some starting locationwithin the virtual reality environment or may be opened and displayed atany convenient location within the virtual reality environment whenactor 101 issues a command “summoning” virtual control panel 127.Tracker-sensors 119 track the location of glove 207, best shown in FIG.2, and, thus, the virtual hand in the virtual reality environment andcompare the location of the virtual hand in the virtual realityenvironment to the locations of the virtual control panel's controls inthe virtual reality environment. When a collision is detected betweenthe virtual hand and a virtual control of virtual control panel 127, thevirtual hand is deemed to be touching the control. Motion captureenvironment 113 responds to the motion of the virtual hand and a mappingof a control state to a desired action causes the desired action tooccur, just as if a physical or real hand had manipulated a physical orreal control. Actor 101 can operate a virtual control of virtual controlpanel 127 in the same way actor 101 can physically operate a tangible,physical object or control capable of being physically touched andphysically manipulated. It should be noted that touching buttons, knobs,switches, and the like of the virtual control panel is but one way ofinteracting with the virtual control panel.

Moreover, virtual control panel 127 can grow and shrink in size andcapability without limit. Furthermore, virtual control panel 127 can bemade to disappear or reappear at the will of actor 101, withoutinterfering with the scene in the virtual reality environment. Virtualcontrol panel 127 is able to float at any location and orientationdesired by actor 101.

The interactive display of data in motion capture environment 113provides many advantages to a virtual reality experience. For example,the display of data in three-dimensional space is more intuitive andallows the user to see phenomena that may be hidden in two-dimensionalrepresentations of three-dimensional data. Moreover, the display ofthree-dimensional data in three-dimensional space makes observing allsurfaces of the object easier. Surfaces that may be difficult orimpossible to see in a real world setting are more easily inspected in avirtual environment. Furthermore, virtual objects are infinitelyre-orientable and scalable by the actor in the virtual environment, sothat experts are more closely integrated with the analysis. Virtualscenes incorporating analysis data provide the potential to linkanalytic solutions to displays for interactive virtual experimentation.

It should be noted that motion capture environment 113 comprises one ormore computers, such as computer 115, executing software embodied in acomputer-readable medium that is operable to produce and control thevirtual reality environment. The scope of the invention encompasses,among other things, motion capture environment, such as motion captureenvironment 113 of FIG. 1; the software operable to produce and controlthe virtual reality environment; and the method for producing andcontrolling the virtual reality environment, carried out by motioncapture environment 113.

The particular embodiments disclosed above are illustrative only, as theinvention may be modified and practiced in different but equivalentmanners apparent to those skilled in the art having the benefit of theteachings herein. Furthermore, no limitations are intended to thedetails of construction or design herein shown, other than as describedin the claims below. It is therefore evident that the particularembodiments disclosed above may be altered or modified and all suchvariations are considered within the scope and spirit of the invention.Accordingly, the protection sought herein is as set forth in the claimsbelow. It is apparent that an invention with significant advantages hasbeen described and illustrated. Although the present invention is shownin a limited number of forms, it is not limited to just these forms, butis amenable to various changes and modifications without departing fromthe spirit thereof.

The invention claimed is:
 1. A system, comprising: an analysis systemfor performing an analysis and generating an analysis data associatedwith the analysis; and a motion capture environment interfaced with theanalysis system, the motion capture system comprising: at least onesensor-tracker for tracking a location of a tracked object within themotion capture environment; and one or more computers collectivelyoperable to generate a virtual reality environment, the virtual realityenvironment being generated by a studio, the studio being athree-dimensional space for containing the motion capture environment,an actor, and the tracked object, the one or more computers beingfurther collectively operable to display the virtual reality environmentto the actor that is physically present in the studio; wherein the oneor more computers are further collectively operable to generate anddisplay the analysis data associated with the analysis to the actor sothat the actor can interact with a virtual representation of theanalysis data in the virtual reality environment; wherein the analysissystem interacts with the one or more computers through an interface toprovide a virtual representation of the analysis data to be observed,the analysis system is configured to permit virtual modification of theanalysis data in real time to achieve an interactive virtualexperimentation; wherein the actor interacts with the analysis system todetermine conditions of the analysis; wherein the tracked object isrelated to the actor and the one or more computers are furthercollectively operable to allow the actor to modify an element beinganalyzed by the analysis system; and wherein modification of the elementby the actor is accomplished by placing a virtual tuning weight on theelement being analyzed, the virtual tuning weight being selected by theactor from among a plurality of virtual tuning weights in a virtualtoolbox.
 2. The system, according to claim 1, wherein the analysissystem includes a computational fluid dynamics system.
 3. The system,according to claim 1, wherein the analysis system includes acomputational stress analysis system.
 4. The system, according to claim1, wherein the analysis system includes a finite element analysissystem.
 5. The system, according to claim 1, wherein the one or morecomputers are further collectively operable to: generate a virtualreality environment including a virtual control panel having a virtualcontrol that, when actuated, effects a predetermined result in thevirtual reality environment; determine a virtual location of the trackedobject within the virtual reality environment; and determine when thevirtual location of the tracked object coincides with the location ofthe virtual control to actuate the virtual control.
 6. A method forinteractively displaying data in a motion capture environment, themethod comprising: performing an analysis through an analysis systemconfigured to communicate through an interface with one or morecomputers, the analysis system generating an analysis data associatedwith the analysis; generating a virtual reality environment includingthe analysis data associated the analysis, the virtual realityenvironment being generated by a studio, the studio being athree-dimensional space for containing the motion capture environment,an actor, and the tracked object, the one or more computers beingfurther collectively operable to display the virtual reality environmentto the actor that is physically present in the studio; and displayingthe virtual reality environment to an actor participating in the virtualreality environment; and experimenting with the analysis data todetermine solutions; wherein the one or more computers are operable togenerate and display the analysis data associated with the analysis tothe actor so that the actor can interact with a virtual representationof the analysis data in the virtual reality environment; wherein theanalysis system is configured to permit virtual modification of theanalysis data in real time to achieve real time solutions through aninteractive virtual experimentation; wherein the analysis system isconfigured to permit modification of the analysis system by the actor;wherein experimenting with the analysis data includes: modifying anelement in the virtual reality environment analyzed by the analysissystem; and performing an analysis through the analysis system for themodified element; and wherein modifying the element includes placing avirtual tuning weight on the element being analyzed, the virtual tuningweight being selected by the actor from among a plurality of virtualtuning weights in a virtual toolbox.
 7. The method, according to claim6, wherein performing the analysis includes performing a computationalfluid dynamics analysis.
 8. The method, according to claim 6, whereinperforming the analysis includes performing a computational stressanalysis.
 9. The method, according to claim 6, wherein performing theanalysis includes performing a finite element analysis.
 10. The method,according to claim 6, further comprising: generating a virtual controlpanel within the virtual reality environment, the virtual control panelincluding a virtual control that, when actuated, effects a predeterminedresult in the virtual reality environment; determining a virtuallocation of a tracked object within the virtual reality environment thatis associated with the actor; and determining when the virtual locationof the tracked object coincides with the location of the virtual controlto actuate the virtual control.
 11. The method, according to claim 10,further comprising: providing information relating to the analysis tothe actor via the virtual control panel.
 12. Software for interactivelydisplaying analysis data in a motion capture environment, the softwareembodied in a computer-readable medium and when executed operable to:perform an analysis through an analysis system in communication with oneor more computers and generate the analysis data associated with theanalysis; generate a virtual reality environment including results ofthe analysis; display the virtual reality environment to an actorparticipating in the virtual reality environment; the virtual realityenvironment being generated by a studio, the studio being athree-dimensional space for containing the motion capture environment,an actor, and the tracked object, the one or more computers beingfurther collectively operable to display the virtual reality environmentto the actor that is physically present in the studio; generate anddisplay the analysis data associated with the analysis to the actor sothat the actor can interact with a virtual representation of theanalysis data in the virtual reality environment and the actor canadjust conditions the analysis is based upon; wherein the analysissystem is configured to permit virtual modification of the analysis datain real time to achieve real time solutions through an interactivevirtual experimentation; the software being further operable to: modifyan element in the virtual reality environment analyzed by the analysis;and perform an analysis for the modified element; wherein modifying theelement includes placing a virtual tuning weight on the element beinganalyzed, the virtual tuning weight being selected by the actor fromamong a plurality of virtual tuning weights in a virtual toolbox; andwherein the computer-readable medium is non-transitory.
 13. Thesoftware, according to claim 12, wherein the analysis includes acomputational fluid dynamics analysis.
 14. The software, according toclaim 12, wherein the analysis includes a computational stress analysis.15. The software, according to claim 12, wherein the analysis includes afinite element analysis.
 16. The software, according to claim 12,further operable to: generate a virtual control panel within the virtualreality environment, the virtual control panel including a virtualcontrol that, when actuated, effects a predetermined result in thevirtual reality environment; determine a virtual location of a trackedobject within the virtual reality environment that is associated withthe actor; and determine when the virtual location of the tracked objectcoincides with the location of the virtual control to actuate thevirtual control.
 17. The software, according to claim 16, furtheroperable to: provide information relating to the analysis to the actorvia the virtual control panel.